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ABSTRACT 


In this paper, an adaptive multimodal biometric fusion algo- 
rithm is proposed. It is based on belief functions and Par- 
ticle Swarm Optimization (PSO). The fusion is performed 
at the score level using belief functions such as Demp- 
ster Shafer, Yager, Proportional Conflict Redistribution and 
Dezert-Smarandache hybrid rules. A hybrid PSO is employed 
to select the best belief function and estimate its parameters. 
Several experiments have been conducted on BANCA dataset 
and a comparison between the well established methods has 
been performed. The preliminary results provide adequate 
motivation towards future research in the application of opti- 
mization techniques in the belief functions. 


Index Terms— Biometric fusion, score level fusion, be- 
lief functions, PSO 


1. INTRODUCTION 


Biometrics is a statistical measurement of human physiolog- 
ical/behavioural traits. It can be used as an alternative of 
the traditional security systems based on keys, cards, badges, 
passwords or PIN numbers. Unimodal biometric systems rely 
on a single modality, so they are limited against accuracy and 
vulnerability to spoofing. This is mainly due to many reasons 
such as imperfect sensor, noisy data, intra-class variation and 
non universality [1]. To overcome these limitations, the fu- 
sion of biometric systems has been proposed [1]. Multi bio- 
metric systems combine various biometric data at different 
levels like sensor level, feature extraction level, score level 
or decision level. The fusion at score level is widely used in 
biometric as it is simple and efficient. It is based on the com- 
bination of similarity scores of the biometric matchers. Fu- 
sion methods at score level are divided into three categories 
[2] statistical, learning and belief functions based methods. 
Statistical techniques combine the scores of the different uni- 
modal matchers by using various basic statistical rules such 
as sum, product, Max and Min. Learning techniques clas- 
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sify multimodal scores into one of the two classes: genuine 
or impostor. The main techniques are support vector machine 
(SVM), Bayesian inference and neural networks (NN). Be- 
lief functions are used to convert the scores into beliefs as- 
signments which are mixed by a combination rule based on 
Dempster Shafer and Dezert-Smarandache theories. 

The main problem with statistical and learning fusion 
techniques appears when different unimodal biometric sys- 
tems produce highly conflicting results. These methods are 
not able to handle this conflict and the fusion performance 
is not enhanced. In opposition, belief functions can manage 
the conflict between many unimodal biometric systems [2]. 
Many belief functions have been proposed in literature. Each 
function differs from another one and the most important 
problem is how to choose the belief function which gives the 
best performance. Motivated by this case, we propose to use 
a hybrid PSO to select the best belief function and estimate its 
parameters in order to obtain a higher accuracy. PSO is used 
because it is a powerful method for parameters estimation. 

The remainder of this paper is divided into four sections. 
Related work is discussed in section 2, while section 3 de- 
scribes the proposed method. Experimental results are pre- 
sented in section 4. The paper is concluded in section 5. 


2. RELATED WORK 


Recently, the design and the development of a multimodal 
biometric system which automatically selects the best fusion 
rule and estimates its parameters has become one of the most 
active research area. In literature, some works have been pro- 
posed. 

An adaptive multimodal biometric fusion algorithm has 
been proposed by Veeramachaneni et al. [3]. It is based on 
a Bayesian decision fusion and PSO. A Bayesian framework 
has been employed to combine the decisions of several bio- 
metric classifiers. PSO has been used to search the best deci- 
sion fusion rule and the threshold at a desired security level. 
This algorithm has been tested only on simulated data to in- 
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vestigate its performance. The simulated data distribution is 
assumed to be Gaussian which is not true for several biomet- 
ric systems. 


A method for enhancing the performance of correlated 
biometric classifiers is suggested by Srinivas et al. [4]. It 
is based on the weighted sum rule and PSO. PSO has been 
used to compute the weight for each classifier. A Bayesian 
risk function is used as a fitness function. This approach has 
been tested on the NIST BSSR dataset and on synthetic scores 
which have been generated by a multivariate normal distribu- 
tion. This method outperforms the classical weighted sum 
rule. 


A Particle Swarm Optimization scheme in the weighted 
sum rule is proposed by Anzar et al. [5]. The d-prime statis- 
tics has been used to measure the separation between the gen- 
uine and the impostor score distribution. It is calculated for 
both of fingerprint and voice modalities. The weight of each 
modality is based on the ratio of these two statistics and it is 
estimated by PSO. This method has been studied under var- 
ious noise conditions. It has decreased the FAR (False Ac- 
ceptance Rate) even at low conditions. The recognition rate is 
enhanced above 0 dB SNR (Signal to Noise Ratio). However, 
this method presents poor results under noise conditions (for 
SNRs<0 dB). 


An evolutionary approach for adaptive combination of 
multiple biometric systems is presented by Kumar et al. [6]. 
This adaptive combination consists on finding an optimal 
fusion rule and estimating its parameters by using a hybrid 
PSO. The score level fusion rules used in this work are sum, 
product, exponential sum and tanh hyperbolic sum. This ap- 
proach has been tested on real and simulated biometric data. 
Experiments have shown that this approach achieves good 
and stable performance over the fusion at the decision level 
based on PSO. 
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3. PROPOSED METHODOLOGY 


In the previous mentioned works [3], [4], [5] and [6], the in- 
tegration of PSO in a multi biometric system to choose the 
best fusion rule and estimate its parameters has given promis- 
ing results. These works have only used combination rules 
such as: or, and, weighed sum, product, exponential sum and 
tan-hyperbolic sum rules. To our knowledge, there is no work 
which uses PSO combined to other fusion techniques such as 
learning and belief functions methods. Consequently, in this 
work, we propose the fusion of unimodal biometric systems 
for person verification using belief functions and evolution- 
ary method. This choice is justified by the fact that the be- 
lief functions can deal with the conflict between the different 
classifiers and the evolutionary techniques like PSO allows 
the best parameters estimation. 

The proposed method consists in the following steps as 
depicted on figure 1. First, the score of each biometric system 
is transformed into belief assignment by using the best pa- 
rameters obtained by PSO. Next, the fusion is performed by 
the best belief function returned by PSO. Finally, in the deci- 
sion step, a person is classified as a genuine or an impostor by 
using the best decision threshold calculated by PSO. 


3.1. Transformation of the scores into masses 


Typically, the classification step in a verification biometric 
system is formulated as a two class problem. The two classes 
are genuine Ogen and impostor imp. Consequently, © = 
{9imp, limp} is used as a frame of discernment. 

In this step, the score provided by each unimodal biomet- 
ric system is transformed into three masses: the mass of gen- 
uine Ogen, the mass of impostor imp and the mass of the un- 
certainty Oimp U Gimp. This transformation is performed with 
Appriou [7] model. It is defined by: 
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Fig. 1. Flow diagram of the proposed method. 


Mi (Ogen) = aj x W (siz) 
Mi (Gimp) = ai x (1— Y(siz)) (1) 
Mii (9) = 1l— Qi 


Where 
i corresponds to the i*” unimodal biometric system. 
a, is the confidence factor of the it” unimodal biometric 
system such as 0 < a; < 1. 

Sij is the match score of a person j provided by the qn 
unimodal biometric system. 

w is an increasing function which maps the scores in the 
range [0, 1]. The sigmoid function is used in the proposed 
method. It is defined by: 


W(siz) = logsig(siz) = 1/(1 + exp(—si;)) (2) 


Computing (a1, 2, .... QN) is performed by PSO in section 
3.4 where N is the number of unimodal biometric system. 


3.2. Score level fusion by belief functions 


Belief functions theory is a powerful tool for representing un- 
certain knowledge. It is a generalization of the probability 
theory. It includes many approaches. In this work, Demp- 
ster Shafer, Yager, Proportional Conflict Redistribution and 
Dezert-Smarandache hybrid combination rules are used. 

The Dempster Shafer (DS) combination rule of two belief 
functions m1(.) and m2(.) over the power set of © (i.e. 2°) 
is defined by [8]: 


J x Ye, xnY=A mı(X)m2(Y) 


mps(A) = 1— kiz 


(3) 





ky2 = my(X)m2(Y) (4) 
X,YE2°, XNY=0 

Where 

AE 29 = {9gen; limp, Ogen U bimp} 

1 — kı2 is a normalizing factor, it consists on eliminating 
the conflict of information between the sources to combine. 

m;(.) represents the belief assignment of the it” unimodal 
biometric system. 

The Yager combination rule admits that in the case of con- 
flict the result is not reliable. In this case, the factor kıs is 
considered as an absolute discounting term added to the igno- 
rance. This rule is given by [2]: 


My ager (9) =0 
My ager (A) = Dox,Ye2®,xny=A my (X)m2(¥) (5) 
My ager(9) = m,(9)m2(0) + kı2 


The Proportional Conflict Redistribution (PCR) combina- 
tion rule redistributes the partial conflicting mass to the el- 
ements involved in the partial conflict. It has five versions: 
PCR1 to PCRS. In [2], PCR5 is considered as the most ac- 
curate version. The PCR5 combination rule of two sources is 
given by [2]: 


mpcrs(A) = m12(A)+ 
5 | My (A)?m2(B) m2(A)?m1(B) 
A,BEe2°,ANB=0 mı (A) + mə(B) mg (A) af m(B) 





Where m12(A) corresponds to the conjunctive consensus 
on A between the two sources. It is defined as: 


m12(A) = > my(X)m2(Y) (7) 
X,YE2°,XNY=A 
The basic idea of the Dezert-Smarandache hybrid com- 
bination rule (DSmH) is to define the belief assignments on 
hyper-power set D° = AO says limp, Ogen N Bimp, Ogen U 
bimp} The DSmH combination rule of two independent 
sources of evidence is defined as follow [2]: 


MDSmH (0) = 0 
mpsmH(A) = Dix yepe xny=4 M(X )ma(Y)+ 
Xx yeo auv uep M1 (X)mea(¥)+ 
x,Yep®,xuy=a,xnyeg M(X )m(Y) 
Where 
U is the disjunctive form of X N Y 
I is the total ignorance. 


(8) 


3.3. Decision 


In this step, the fused beliefs are transformed into a probabil- 
ity measure by using the pignistic transformation [2] in the 
case of DS, Yager and PCRS rules and the generalized pignis- 
tic transformation [2] in the case of DSmH rule. These two 
transformations are defined by (9) and (10). Then, a statistical 
classification approach such as the likelihood ratio test is used 
to compute the final decision (see (11) and (12)). 
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Where |Y | denotes the cardinality of Y. 
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Where Cm (Y) is the DSm cardinality corresponding to 
the number of parts of Y. 
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(12) 
Where A is the decision threshold that minimizes the 
weighted error rate (WER) on a development set. 


3.4. Belief function and confidence factor optimization 


PSO is a population optimization method. It was proposed by 
Kennedy and Eberhart in 1995 [9]. A particle in the search 
space is considered as an individual bird. It has an initial 
position and velocity. To find the best solution, all particles 
and their velocities are replaced by the best previous position 
of the current particle Pbest; and the best previous position 
of all particles Gbest using (13) and (14). A fitness function 
is used to evaluate the position of the particles. After a limited 
number of iterations, the particle that satisfies the global best 
fitness is chosen as the optimal result. 


u(t +1) =w x vilt) +c x rı x (Phest; — p;(t)) 
+c2 X r2 x (Gbest — p;(t)) 
(13) 


pilt +1) = pi(t) + vi(t + 1) (14) 


Where 

v; is the velocity of the it” particle. 

pi is the position of the it” particle. 

w is the inertial weight. 

cı and C2 are the acceleration constants used to pull each 
particle towards Pbest and Gbest. 

rı and r2 are random numbers generated between [0, 1]. 

Equation (14) operates on continuous space. However, 
there exist optimization problems where the particles are rep- 
resented in a discrete space. The concept of PSO in a discrete 
domain is proposed in [10]. It is named binary PSO (BPSO). 
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In BPSO, (14) is rewritten as follows: 


_f 1 if rg < S(u(t¢+1)) 
BED T= { 0 otherwise >) 

Where r3 is a random number generated between [0, 1] 
and S is the sigmoid function. 

In our implementation, BPSO is employed to select the 
best belief function among the following functions: DS, 
Yager, PCRS and DSmH. Each particle is composed of logo K 
binary values, where K is the number of belief functions. 
However, PSO is used to estimate the confidence factors (a, 
Q2, ..., QN) with the constraint 0 < a; < 1, fori = 1,2,..,N 
where N is the number of modalities. The flow diagram of 
this step is explained in figure 2. 

At the initial step, the particles population is set to 10 and 
the maximum number of iterations to 100. After randomly 
initializing the position and velocity of all particles, all train- 
ing scores are transformed into belief assignments. Then, they 
are multiplied by the position of PSO particles. After that, the 
belief assignments are fused by the function which exists in 
BPSO particles. The weighted error rate (WER) is used as a 
fitness function. It is defined by [11]: 


WER(A) = CpaxFAR(A)+(1-Cpa)xFRR(A) (16) 


Where Cr, varies from 0 to 1, it balances between the 
costs of FAR and FRR. FAR is the false acceptance rate 
and F RR is the false rejection rate. 

At the end of iterations, Gbest contains the best belief 
function and its parameters (confidence factors). 


Calculate FFR, FAR and WER by varymg the | 
threshold over the whole range of the fused score | 


Fig. 2. Flow diagram of particle swarm optimization step. 


4. RESULT 


The experiments have been conducted on the scores of the 
BANCA multimodal base [12]. It contains real bimodal face 
and speech data. The English subset [11] contains 52 sub- 
jects. It is divided into two sets, called g1 and g2. Each set 
contains 13 males and 13 females. g1 is used as a develop- 
ment set and g2 is used as an evaluation set. In the BANCA 
database, there are 7 different protocols [11] matched con- 
trolled (Mc), matched degraded (Md), matched adverse (Ma), 
unmatched degraded (Ud), unmatched adverse (Ua), pooled 
test (P) and grant test (G). 

The PSO parameters c1, C2 and w are computed by the 
formula cited in [13] and [14] respectively. The PSO algo- 
rithm is running 10 times. In each run, the HTER (Half Total 
Error Rate) is computed for each combination of the different 
classifiers on a test set (g2). The HTER is defined as [11]: 


HTER(A*) = (FAR(A*) + FRR(A*))/2 (17) 


Where A* = argminW ER(A). 

The HTER achieved by the proposed approach and the in- 
dividual classifiers is presented in Table 1. The proposed ap- 
proach achieves better HTER than the individual classifiers. 
Also, it is seen that the proposed approach reaches a high ac- 
curacy when compared to the fusion based on belief functions 
without using PSO over all the BANCA protocols. 

Table 1 presents also the most selected function by BPSO. 
This selection differs from one protocol to another. For ex- 
ample, Yager rule is the most selected function in Ua and Ma 
protocols. However, PCR5 rule is the best function in Ud 
protocol. So, we confirm that using BPSO to select the best 
fusion rule is very important to improve the verification accu- 
racy. 

Table 2 summarizes a comparative study between the pro- 
posed method, the methods cited in [3] and the SVM based 
fusion using RBF kernel. The proposed approach surpasses 
the SVM based fusion on all the protocols because the be- 
lief functions deal the conflict between the classifiers but the 
SVM did not handle such conflict. Also, it is noticed that 
the proposed method outperforms the fusion at decision level 
using PSO [3] on all the protocols because the fusion at the 
score level has more significant information compared to the 
fusion at the decision level. 

Figure 3.a presents the confidence factors calculated by 
the proposed approach by varying Cra. We notice that 
the confidence factors values change by varying Cpa. This 
proves that the proposed approach find the best confidence 
factors that achieve the best performance. 

Figure 3.b presents the probability of selecting belief 
function by varying CF4. It is noticed that DS rule is the best 
function for 0.1 < Cra < 0.4. However, PCRS rule is most 
selected rule for 0.5 < Cra < 0.6 and 0.8 < Cra < 0.9. 
Moreover, Yager rule is the best one for 0.7 < Cra < 0.8. 
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Fig. 3. (a) Confidence factors achieved by the proposed ap- 
proach, (b) Probability of selecting belief function, (c) HTER 
computed by varying Cp 4 using G Protocol. 


Figure 3.c presents the HTER computed by the differ- 
ent methods against Cr 4 variations. It is noticed that SVM 
based fusion is the best method for 0 < Cpa < 0.2 and 
0.9 < Cra < 1. However, the proposed approach yields the 
minimum values of HTER for 0.2 < Cra < 0.9. 


5. CONCLUSION 


In this study, a new adaptive multi biometric fusion algorithm 
is proposed. It is based on belief functions and particle swarm 
optimization. The contribution of this research consists on 
selecting the best belief function rule among DS, Yager, PCR5 
and DSmH by a binary PSO and estimating its parameters by 
PSO. 

The comparison of the proposed approach with the fu- 
sion at decision level using PSO and the SVM has shown that 
the proposed method outperforms these methods on all the 





























Protocol | Face Voice DS PCRS | Yager | DSmH | The proposed method | The most selected function 

G 7.146 | 4.082 1.447 1.727 1.720 1.718 1.181 DSmH 
P 18.480 | 8.412 | 6.779 | 7.548 | 7.573 | 7.543 6.100 DS 

Ua 28.171 | 15.159 | 12.704 | 14.336 | 14.377 | 14.302 11.549 Yager 

Ud 16.564 | 3.876 | 3.744 | 4.643 | 4.675 | 4.633 2.285 PCRS5 

Ma 12.706 | 11.785 | 6.498 | 6.481 6.462 | 6.459 6.249 Yager 

Mc 3.835 | 2.962 1.101 1.241 1.252 1.270 1.100 DS 

Md 8.459 | 6.244 | 2.125 | 2.744 | 2.738 | 2.752 2.676 DS 














Table 1. Average HTER of the individual classifiers, DS, PCR5, Yager, DSmH and the proposed method (Cr 4 = 0.5). 

















Protocol | The proposed method | Fusion at decision level based on PSO | SVM (RBF kernel) 

G 1.181 2.402 1.325 
P 6.100 7.637 6.266 

Ua 11.549 14.235 12.104 

Ud 2.285 4.119 3.095 

Ma 6.249 8.888 6.459 

Mc 1.100 1.785 1.334 

Md 2.676 3.797 2.960 











Table 2. Average HTER of the proposed method, fusion at decision level based on PSO and SVM based fusion (CF 4 = 0.5). 


BANCA protocols. 


In future work, we propose to integrate other fusion tech- 


niques in the proposed framework in order to select the fusion 
technique which gives the best performance. 
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